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We compute the entropy reduction in feedback controlled systems due to the repeated operation 
of the controller. This was the lacking ingredient to establish the thermodynamics of these systems, 
and in particular of Maxwell's demons. We illustrate some of the consequences of our general results 
by deriving the maximum work that can be extracted from isothermal feedback controlled systems. 
As a case example, we finally study a simple system that performs an isothermal information-fueled 
particle pumping. 
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I. INTRODUCTION 

Controllers are ubiquitous in science and technology 
with a number of purposes such as stabilizing unstable 
dynamics or increasing the performance [if]. Further- 
more, many real systems in nature can be modeled as a 
system plus a controller. A controller is an external agent 
whose action is to modify the evolution of the system 
with a purpose. Feedback or closed-loop controllers use 
information about the state of the system. The feedback 
is the process performed by the controller of measuring 
the system, deciding on the action given the measure- 
ment output, and acting on the system. On the contrary, 
an open-loop controller operates on the system blindly, 
i.e., without information of its state. Although it is intu- 
itively clear that the information about the state of the 
system can be used to improve the performance, there are 
still open questions on the connections between feedback 
control theory and information theory (see Ref. [l|). In 
particular, the understanding of the thermodynamics of 
feedback control is still incomplete. Much of the progress 
in the solution of this problem has come from the study 
of Maxwell's demon 0]. This is a being that gathers 
information about a system and is able to decrease the 
entropy of the system without performing work on it. 
The seminal work of Szilard [|[ contains the basic ingre- 
dients of the trade off between information theory and 
thermodynamics, which is precisely stated in Landauer's 
principle: The erasure of 1 bit of information at temper- 
ature T implies an energetic cost of at least fee Tin 2 
Bennett [5] pointed out that Landauer's principle is the 
key to preserving the second law of thermodynamics in 
feedback systems, as the controller must erase its mem- 
ory after each cycle to allow the whole system to truly 
operate cyclically. How to achieve the shorter descrip- 
tion for the memory record of the controller in order to 
minimize the energetic erasure cost was established by 
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Zurek Q by using an algorithmic complexity approach. 
On the other hand, Lloyd used in 0] a different point of 
view — that of the feedback controlled system. From this 
approach the effect of the interaction of the controller 
with the system is to reduce the entropy of the system, 
due to the additional determination of the macrostate 
of the system through the information obtained from it. 
More recently, Touchette and Lloyd 0] have computed 
the maximum additional reduction in entropy attainable 
in one control action when a feedback control is used in- 
stead of an open-loop control. 

In this paper we also consider the point of view of 
the feedback controlled system. The thermodynamics of 
the interactions of the system with the controller and 
the environment are well studied for the heat and work 
exchanges. However, a complete understanding of the 
entropy reduction in the system due to its interaction 
with the feedback controller is still lacking. We solve 
here this problem and show how to compute this entropy 
reduction after one or several control steps. This result 
allows us to establish the thermodynamics of feedback 
controlled systems without assuming Landauer's princi- 
ple. Several concepts and quantities defined in informa- 
tion theory Q emerge naturally as one computes this 
entropy reduction. For the definition of the entropy we 
will use ks = 1 and natural logarithms. This implies that 
the information quantities that naturally appear will be 
in nats (In 2 nats = 1 bit). 

In the next section we compute the entropy reduction 
in a general feedback controlled system due to the re- 
peated operation of the controller. The result allows us to 
establish the thermodynamics of feedback controlled sys- 
tems. In Sec. IIII1 we illustrate some of the consequences 
of our general result by deriving the maximum work that 
can be extracted from isothermal feedback controlled sys- 
tems. In Sec. IIV1 we show the applicability and usability 
of the results in a simple dynamical system, a Markovian 
particle pump that is able to extract useful work from 
the entropy reduction due to the information used by an 
external feedback controller. Finally, we summarize the 
results of the paper in Sec. [V] 
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II. ENTROPY REDUCTION IN FEEDBACK 
CONTROLLED SYSTEMS 

Let us call X k := X(tk) the macrostate of a general dy- 
namical system at the fcth control step of the controller 
(at time t k ). In a feedback controlled system the control 
step involves several operations by the controller: mea- 
suring the system, deciding the control action to take 
given the measurement output, and acting on the sys- 
tem following the selected control action. Therefore, the 
control action is the modification of the evolution of the 
system made by the external agent that we shall call the 
controller. The controller can perform several control ac- 
tions on the system. By C\ — c we denote that, at the 
first control step, the controller has chosen to perform 
the action labeled by c. (It is not a specification of the 
state of the controller.) As the control actions are decided 
at their respective control steps, C' k represents only the 
decision taken at the fcth control step. 

Initially the entropy of the system is So, which is fixed 
by the probabilities px (x) of each possible microstatc x 
at time t = 0. Subsequently, the system evolves with an 
entropy change from So to Sj, which is the entropy just 
before the first control step. It is given by the statistical 
entropy 

S i = - E Px 1 ( x ) l *PXi (*) = : H (!) 

x^X 

with 8£ as the set of possible microstates of the system. 
At time t\ the controller measures the state of the system. 
The result of this measurement determines, at least par- 
tially, the action the controller will take. The additional 
information on the system provided by the measure fur- 
ther determines the system macrostate Q, i.e., it defines 
a submacrostate that contains only microstates compat- 
ible with the measured value. However, from the point 
of view of the system, each set of measurement outputs 
that leads to the same control action can be considered 
as defining a single submacrostate of the system, because 
the controller in its action on the system ignores the dif- 
ferences inside these sets. Thus, if the measurement im- 
plies a control action C\ = c, the entropy of the system 
decreases to 

ff(*i|Ci =c):=-J2 Px 1 \c 1 (x\c)]np Xl[Cl (x\c)- (2) 

Therefore, the average entropy after the first control step 
can be obtained by averaging over the set of all possible 
control actions, 

S i = E^cM^iICi = c) =: HiX^Ci). (3) 

Hence the average variation in the entropy at the first 
step is 

AS, = St - Si = tf(Xi|Ci) - H{X X ) =: -J(*i;Ci), 

(4) 



i.e., it is the (minus) mutual information [9( between X\ 
and C\. 

Let us describe one more step. Each of the previous 
submacrostates of the system with entropy H{X\\C\ — 
c) evolves to give an entropy H{X-i\Ci — c) just before 
the second control step. Following the second control 
step, each one of these submacrostates of the system give 
\&\ more submacrostates. The entropy of the system 
given that C\ = c and C 2 = c' is H(X 2 \C 2 = c 1 ,C\ = c). 
Therefore, the average entropy of the system after the 
second step is 

S a 2 = E PccAc, c)H(X 2 \C 2 = c', Ci = c) 

c.c'G^ (5) 

= H(X 2 \C 2 ,C 1 ), 

and the average variation in the entropy at this sec- 
ond control step is AS 2 = S$ - 5 2 b = H(X 2 \C 2 ,d) - 
H{X 2 \d) = -I{X 2 -C 2 \d). This conditioning of the 
mutual information shows that the entropy of the sys- 
tem is only reduced by the new information. 

Analogously we get for the average entropy reduction 
in the fcth step AS k = -I{X k ;C k \C k - v ), where C k ~ 1 
stands for C k -i, C k -2, ...,C\. Using the properties of 
mutual information [9(, this average entropy reduction 
can be written as 

AS k = -I{X k -C k \C k - 1 ) = -I{C k -X k \C k - 1 ) 

= -HiCklC*- 1 ) + H(C k \C k -\X k ). !( " 

Finally, we find that the total average entropy reduc- 
tion due to the information used in M control steps is 
AS'info = Dfeti AS fc , i.e., 

M 

AS info = -Y / HC k ;X k \C k - 1 ). (7) 

fe=i 

This general result indicates that this entropy reduction 
can be computed in terms of the joint probabilities for 
the state of the system and the control actions history. 
Using Eq. © and the chain rule for H (see Ref. Q), we 
rewrite the last equation as 

M 

AS info - -H(C M ) + J2H(C k \C k -\X k ). (8) 

fe=i 

Equation 0, or equivalently Eq. is a central result 
of this paper. As a consistency check, note that for open- 
loop controlled systems the controller acts independently 
of the state of the system and it gets no information of 
it. Thus, H(C k \C k -\X k ) = H{C k \C k - 1 ), which gives 
ASi n fo = after applying the chain rule in Eq. ([8]), as ex- 
pected. Note also that the mutual information in Eq. ([7]) 
between the system and the control actions is conditioned 
by the past control actions. This reflects that the corre- 
lations between measurements limit the attainable en- 
tropy reduction. Therefore, the entropy reduction in M 
consecutive measurements is equal or lower than in M 
independent measurements. 
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A. Deterministic feedback controllers 

A relevant class of closed-loop controllers is determin- 
istic feedback controllers. For them the control action is 
determined without uncertainty by the state of the sys- 
tem and the control actions history. Therefore 



H(C k \C k -\X k )=0, 



(9) 



and the entropy reduction in Eq. ([5]) simplifies to 
A<5i n f = —H(C M ), which can be computed by just us- 
ing the joint probability pCi,...,C M ( c 1j ■ ■ ■ i c m)- Conse- 
quently, the average entropy reduction after a large num- 
ber of control actions is given by the entropy rate H( c <a) 
of the stochastic process describing the control actions: 



lim 



AS 



info 



M- 



lim 

Af->o> 



-H(C M ) 
M 



-H(n (io) 



For a system and control dynamics without explicit de- 
pendencies in time, this average entropy reduction co- 
incides with the asymptotic entropy reduction in one 
step [3, that is, lim^/^oo A5 lnfo /M = liniM^^ AS M - 



B. Non-deterministic feedback controllers 

Feedback controllers satisfying Eq. ([9]) are error free. 
On the other hand, controllers affected by some source 
of error are common in real systems. In this case the 
decorrelation between the control actions and the state 
of the system reduces the attainable entropy reduction; 
see Eq. For instance, consider a feedback controller 
with two possible actions, say "on" and "off", for which 
the system state and the previous control actions history 
determine which one of the actions is taken with proba- 
bility 1 - e. For this system, ^(CfclC^ 1 , A fe ) = H b (e), 
with Hb(e) as the binary entropy function -ff&(e) := 
-elne - (1 - e) ln(l - e), and Eq. © gives 



lim 



ASi, 



ifo 



M 



-H{tf) + H b {e). 



(11) 



This shows that errors in the control operation limit the 
attainable entropy reduction. 



C. Discussion 

The new relation ([7|) sets the entropy reduction in the 
controlled system due to the information used by the 
external agent that operates on it. The reformulation 
of this relation as Eq. ([8]) allows us to understand the 
average entropy reduction per control step as two com- 
peting contributions: a negative term accounting for the 
entropy rate of the control actions, and a positive term 
accounting for the decorrelation between the controller 
actions and the state of the system. This decorrelation 
can arise, for instance, from errors in the operation of the 



controller [see Eq. (|TTj) ]. These new relations, Eqs. ([7]) 
and ijHJ), also show how the past control action history 
must be taken into account to avoid redundancy in the 
computation of the entropy reduction. They are con- 
sistent with the Zurek's computational interpretation of 
the controller as a memory record whose blocks occupied 
by past measurements must be compressed before the 
erasure process [f| [Tpj . On the other hand, when only 
one control step is considered, Eq. ([7]) reduces to Eq. Q), 
which gives the well-known Landauer's energetic cost due 
to information @], kBTI(Xi\Ci) (recovering units), also 
found for quantum systems [111 ]. 

The statement of the entropy reduction in terms of the 
control actions is an important point of this paper. It al- 
lows one to give a reachable bound for the efficiency. (If 
the controller performs the same action for two differ- 
ent measured values, the bound found for the efficiency 
considering the entropy reduction in terms of the mea- 
sure could be nonreachable.) Note also that the overall 
reduction in the entropy of the system due to feedback 
control is expressed in terms of physical quantities and it 
can be computed without knowledge of internal details 
of the controller. In addition, this approach also allows 
one to compute the maximum entropy reduction attain- 
able with a nondeterministic feedback control, Eq. pip , 
giving a reachable bound. 

The entropy reduction in the system due to the infor- 
mation used by the controller is a fundamental ingredient 
in the thermodynamics of feedback controlled systems. 
It is the key to improving the performance in these sys- 
tems compared with their open-loop counterparts. Once 
this entropy reduction is understood and we know how 
to compute it [Eqs. or ©], the thermodynamics of 
feedback controlled systems is complete. In particular, 
we show in the next section how to compute thermo- 
dynamic relations for an isothermal feedback controlled 
system. 



III. APPLICATION: ISOTHERMAL FEEDBACK 
CONTROLLED SYSTEMS 



We study in this section the implications of the pre- 
vious results for the case of an isothermal feedback con- 
trolled system. 

A general isothermal feedback controlled system is a 
system that is coupled to a feedback controller, to a ther- 
mal bath of temperature T, and to another external sys- 
tem on which it does work. When the system is operated 
cyclically, the initial state is recovered after a cycle, and 
the variations in internal energy and entropy of the sys- 
tem in the cycle are zero. During such a cycle the system 
releases a quantity of heat Q to the thermal bath and 
does work W on the external system. The transfer of the 
internal energy of the controller AC/ con t to the system is 
given by the first law of thermodynamics, 



A(7 C , 



Q + W = 0. 



(12) 



On the other hand, the second law of thermodynamics 
gives 



TAS C . 



Q>0, 



(13) 



with AS'cont as the entropy increase in the controller. 
Combining both relations we get the inequality 



W < -AUc, 



(14) 



where A_F cont is the variation in the Hclmholtz free en- 
ergy of the controller in the cycle. From this relation it 
is natural to define the efficiency of a feedback controlled 
system as 



W 



V 



-AF r , 



(15) 



In addition, if the controller only interacts with the 
system and without heat transfer, we have AS'cont > 
— ASjnfo, i.e., the increase in entropy of the controller 
should be greater than or equal to the reduction in the 
entropy of the system due to the actions of the controller. 
This implies that the maximum efficiency that can be at- 
tained with an isothermal feedback controlled system is 



W 



V 



-AUc. 



TAS\ 



info 



(16) 



where W is the work extracted from the system, — A[/ cont 
is the work done by the controller on the system, and 
ASjnfo is the entropy reduction in the system due to the 
information-dependent operation of the controller, which 
can be computed with Eq. ([7]). 



IV. EXAMPLE: MARKOVIAN PARTICLE 
PUMP 

We shall illustrate how to apply our results in a sim- 
ple dynamical system, a Markovian particle pump, which 
is able to extract useful work from the entropy reduc- 
tion due to the information about the system used by 
an external feedback controller. Consider a particle in a 
one-dimensional lattice that is in contact with a thermal 
bath at temperature T. An external controller can acti- 
vate reflecting barriers separated by a distance L with n 
lattice sites between two consecutive barriers; see Fig. [T] 
For the discussion of this example we will consider units 
of fcgT = 1 and L = 1. In the absence of external forces, 
the particle jumps to the left or to the right site with the 
same probability, 1/2, at each time step. Now let us have 
a force / pointing in the negative direction. The prob- 
ability of jumping to the right decreases and becomes 
a := 1/(1 + e^' n ), as follows from detailed balance. We 
aim to move the particle to the right (against the force) . 
For this purpose the controller measures the particle lo- 
cation and consecutively raises from left to right the re- 
flecting barriers to trap the particle further and further 
to the right. The next barrier to the right is raised when 




FIG. 1: Illustration of the Markovian particle pump with 
n = 2 lattice sites between barriers. This is a simple feedback 
controlled system that extracts useful work from the entropy 
reduction due to the information about the system used by 
the external feedback controller. 



the measurement indicates that the particle has crossed 
to the righthand side. This implies that when the par- 
ticle moves to the left until the raised barrier location it 
finds a reflecting boundary condition, while the particle 
has no bounds to its displacements to the right. 

This defines a deterministic feedback control that 
pumps the particle by using information about the lo- 
cation of the jumping particle. We stress that a blind 
open-loop control strategy for the lifting of the barriers 
cannot achieve direct flux against the load. In addition, 
our closed-loop controller does not introduce any extra 
energy in the system. Thus, the entropy reduction in 
the system thanks to the information-gathering opera- 
tion is the only responsible for the pumping. In partic- 
ular, we highlight that a naive definition of efficiency as 
r\ = W/(— AU con t) is meaningless for engines that work 
due to an information-dependent operation. Our general 
results allow us to compute the maximum possible effi- 
ciency of this pump as a case example, not only in the 
quasistatic regime (large time intervals between two op- 
erations of the controller) but also when it is operated 
non-quasistatically (for instance every time step). 

Let us first compute the maximum efficiency attainable 
when the controller operates every time step. We con- 
sider the particle initially at the origin with the reflecting 
barrier to the left raised. At time tj. the controller takes 
the value (7/5 = 1 when the next right barrier is raised 
or Cfc = if the barrier remains off. As the feedback 
control in this example satisfies the deterministic condi- 
tion ([5]), the average entropy reduction per step is given 
by Eq. (|10[) . Furthermore, in order to simplify the com- 
putation of the entropy rate, it is useful to change to a 
description in terms of a new stochastic process C, with 
C s defined as the number of steps between the raise of the 
barrier s— 1 and that of the barrier s (first passage time). 
For example the event (Ci,...,C 7 ) = (0,0,0,1,0,0,1) 
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corresponds to the event (C\, C2) = (4, 3). It is clear that 
we can establish a one-to-one correspondence between C 
and C, as both represent univocally the control actions 
history. Calling (r) as the average first passage time 
through the next barrier position, we have that Eq. (fT0|) 
reads 



lim 

t — >oo 



TAS h 



To 



t 



lim 



lim 



-H(C S ) 



(17) 



[That is, H(tf) = H(^)/{t}.} As the new tilde vari- 
ables are independent and identically distributed we have 
H(C S ) = sH(C 1 ). Thus, 



lim 



TAS inio = -H(C{) = Er=iPr(fc)ln Pr (fc) 

t ( T ) Efell fc Pr(fc) 



(18) 



where p T (k) is the probability mass function of the first 
passage time being r = k. This asymptotic value, 
Eq. (fT8|) . is reached in a characteristic time (r). The 
probability p T (k) can be obtained from the transition 
probabilities between the states of the jumping particle. 

On the other hand, the average potential increase is 
W — //(t). Therefore, the maximum efficiency at- 
tainable at this nonquasistatic regime is obtained from 
Eq. (TIT)]) that reads 



Vnq 



f 



H{C X ) 



(19) 



A. One lattice site between consecutive barriers 

For instance, for the case with a single lattice site 
between two barriers p T {k) = Q>(1 — a)' 0-1 , implying 
H{Ci) — Hb(a)/a and (r) = l/a. Thus, the average 
entropy reduction per step is Hb{a), and the average po- 
tential increase is W = //(t) = af. Finally, the max- 
imum efficiency attainable at this nonquasistatic regime 
is ?7nq = af/Hb(a). This result for the model with a 
single site between two consecutive barriers can also be 
obtained without using Eq. (JTSJ) . For this simple case 
operation steps at different times are independent and 
TAS k = -H(C k ) with pc„(l) = a. This gives an en- 
tropy reduction per step Hb{a). On the other hand, the 
average potential energy gain per step is af because the 
particle gains an energy / with probability a. In view of 
these considerations we recover r) nq = af / Hb(a). 



B. Several lattice sites between consecutive 
barriers 

As a is the probability of jumping to the right, the 
probability of the first passage time being r = k is ob- 
tained from the probability px k _ 1 (n) of finding the par- 
ticle at site n (just to the left to the first barrier) at in- 
stant time k — 1 as p T (k) = apx k _ 1 {n)- To evaluate this 



to 

< 




200 



FIG. 2: Average entropy reduction per time step as a func- 
tion of time for the particle pump with n = 5 lattice sites 
between barriers: numerical simulations (+ signs) and asymp- 
totic value (dashed line) . The asymptotic value is approached 
in a characteristic time of the order of the mean first passage 
time (t). Force / = 1. Units ksT = 1 and L = 1. 



probability we only need to know the transition proba- 
bilities of jumping between the different spatial positions 
(see Fig. [lj. We shall call II as the matrix such that its 
(«,j)th entry is the probability Pj^i of jumping from the 
j site to the i site. Then, for the particle pump with n 
sites between barriers, II is the n x n tridiagonal matrix 



n 



fX — a 1 — a 
a 

a ' ■ 1 — a 



\ 



V 



1 - a 
a / 



(20) 



Assuming that the particle is initially situated at the ori- 
gin, the probability px t _i(i) is given by the (n, l)th el- 
ement of the (k — l)th power of II. Hence, 



p T (k) = aU^in, 1 



(21) 



For instance, for n = 1 we recover p T (k) = a(l — a) k 1 , 
with a = 1/(1 + e*). For n = 2 we get, after 
some straightforward calculus, p T (k) — a(6^.~ 1 — b^ 1 ), 
where a := a 2 j\J\ + 2a — 3a 2 and b± := (1 — a ± 
Vl + 2a-3a 2 )/2, with a = 1/(1 + e^/ 2 ). 

Once the probabilities p T {k) are obtained, the en- 
tropy reduction and the efficiency can be computed with 
Eqs. (TT8"|) and ([TO)) respectively. We plot in Fig. [2] this en- 
tropy reduction lim^oo TASi n { /t for the particle pump 
with n = 5 lattice sites between barriers, together with 
the time dependence of the average entropy reduction 
per time step obtained by means of computer simulations 
of the dynamics in the maximum measurement regime. 
As expected, this time evolution tends to the theoret- 
ical asymptotic value in a characteristic time of order 
( T > = Efcli kp T (k). 
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The numerical results in Fig. [2] have been obtained 
evolving the particle distribution according to the known 
transition probabilities. The entropy reduction in each 
measurement is given by the entropy difference between 
the particle distributions before and after the measure- 
ment. After the measurement we keep one of the two 
possible particle distributions chosen randomly with the 
probability of the corresponding measurement output, 
and we evolve this particle distribution until the next 
measurement. Following this procedure we have per- 
formed several realizations of the control actions history, 
and thereafter we have performed an average over real- 
izations to obtain the average entropy per time step as 
a function of time. For these simulations we have con- 
sidered n — 5 lattice sites and force / = 1 (in units of 
ksT = 1 and L = 1) or equivalently a = 1/(1 + e 1 / 5 ) w 
0.45. 



C. Quasistatic regime 

To conclude the analysis of the illustrating example, 
the Markovian particle pump, we shall compute its max- 
imum efficiency in the quasistatic regime. Consider again 
the particle initially situated at the origin. As the time 
between measurements is large enough, the system has 
reached equilibrium when the controller measures at a 
time t > 1. Hence p Xt ( m ) = i 1 ~ e-f/ n )e~ fm / n and 
the jumping particle is at the righthand side of the next 
barrier with probability ^2 m>n Px t (m) = . On the 
other hand, when the barrier is raised the system gains a 
potential energy /. Thus, the entropy reduction due to 
information is •H^fe - * ), while the potential energy gained 
in one step is fe~f. Therefore the maximum efficiency for 
the quasistatic operation of the Markovian particle pump 
is r; q = fe~l I Ht,{e~f). We note that < ?7 nq < r] q < 1, 
as expected. 

In order to compare with results in Fig. [3] note that 
for the same parameter values a measurement step in 
the quasistatic regime reduces the entropy on average an 



amount H^e" 1 ) w 0.66. However, a measurement step 
in the quasistatic regime requires many evolution time 
steps resulting in a very low entropy reduction per time 
step. 



V. CONCLUSIONS 

In this paper we have addressed the thermodynamics of 
closed-loop controlled systems, focusing on what charac- 
terizes them, namely, the use of information. Our results 
show explicitly how to calculate the entropy reduction 
due to information, Eq. ([7]) or (jHJ). Therefore, they al- 
low one to compute the thermodynamic quantities and 
their relations for feedback controlled systems. In partic- 
ular, we have calculated the thermodynamic relations for 
isothermal feedback controlled systems, Eqs. (fL2"l) - (fl"4")) . 
and also the maximum efficiency attainable, Eqs. (I15p 
and (|16p. As a case example, we have shown how to ap- 
ply our general results to a simple system that performs 
an isothermal information-fueled particle pumping, for 
both a maximum measurement regime and a quasistatic 
regime. The results presented in this paper allow one 
to study the thermodynamics of many other feedback 
controlled systems. It will be particularly interesting to 
obtain the thermodynamics of feedback flashing ratchets 
that have been studied theoretically [ll|, and recently 
realized experimentally [l3j . 
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